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Foreword 



In 1991, the DeWitt Wallace-Reader's Digest Fund initiated a three-year, $3.6 million project 
called CLASP (Connecting Libraries and Schools Project) to promote collaborative programs 
among the New York Public Library, New York City public schools and neighborhood 
organizations. A total of 107 schools from three New York City school districts and 23 
branch libraries were involved in this effort. 

In February 1993, the Fund hired an evaluation team from Information Management 
Consultant Services, Inc. to conduct an evaluation of the final 20 months of the project. While 
the evaluation was more heavily focused on CLASP activities that took place over the summer 
months and less on what transpired during the full academic year, there was sufficient data 
collected for reaching some overarching conclusions about the strengths of the program and 
areas for improvement.* 

The Fund is making the project's final evaluation report and a separate report, "Library 
Collaborations: A L"-.erature Analysis to Aid the Evaluation of the Connecting Libraries and 
Schools Project (CLASP)," available over ERIC to help inform others about the potential for 
similar partnership efforts involving public schools and libraries. 
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EXECUTIVE SUMMARY 

This report presents a summary of the results of the evaluation of the Connecting 
Libraries and Schools Project (CLASP) conducted from February 1993 throi-'gh October 
1994 by Information Management Consultants, Inc. (IMCS). The purpose of the 
evaluation was to assess the degree to which CLASP was successful in reaching its three 
project goals: 

• To establish library use among students at the critical early stages of their 
intellectual development; 

• To organize public librarians, parents, and school personnel in cooperative 
activities that encourage the productive use of library resources; and 

• To develop models for cooperative services and institutional collaboration 
that could be replicated elsewhere. 

This report summarizes and synthesizes the findings, conclusions, and 
recommendations presented throughout the evaluation process and documented in ten 
reports and other documents. For three evaluation studies based on on-site data 
collection, conducted during summer 1993, fall 1993, and summer 1994, this report 
includes the project objectives, design methodology, findings, conclusions, and 
recommendations. In addition, the report describes the initial evaluation plan and the 
selection of sites and instruments. 

This report also contains a summary of other IMCS activities carried out to supplement 
or improve the evaluation effort and to enable CLASP/NYPL to carry the project 
successfully beyond the pilot phase. These activities include, among others, an analysis 
of circulation, workshops for CLASP/NYPL staff, and an in-depth study of school- 
library collaboration. The report ends with our general assessment of CLASP and 
recommendations for improving CLASP as it moves toward self-sufficiency and 
expansion. 

CLASP was remarkable in being one of the first projects to involve cooperation between 
a large urban public library system and public schools in a sustained and systematic 
manner. The evaluation snowed that CLASP had considerable success in reaching the 
first formal project goal, to establish library use among children. For the second goal, to 
organize public librarians, parents, and school personnel in cooperative activities, there 
was some effort but limited success project-wide due to various constraints. CLASP 
made little progress toward the third goal, to develop models for cooperative services 
and institutional collaboration. Despite some limitations, however, it should be noted 
that CLASP did secure funding to carry it beyond its pilot phase; this fact alone attests 
to the success of the project. 
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Overall, the evaluation resulted in. the following conclusions: 

• CLASP programs reached thousands of users who might not otherwise 
have attended the library; these users were highly satisfied with CLASP. 

• The impact of CLASP centered on the number of programs offered, rather 
than the type, and few differences were noted between CLASP and non- 
CLASP library programs. 

• CLASP succeeded in raising schools' and the larger community's awareness 
of the library, although there was general unfamiliarity with CLASP itself. 

• Teachers were enthusiastic about CLASP, but their involvement was not 
systematic or fundamental to program activities. 

• Participants felt CLASP activities were beneficial, and there was some 
evidence of increased library use as a result of CLASP. 

• CLASP succeeded in bringing ethnically diverse and underserved users into 
the library, but programs often failed to create links to books, reading, or 
library services. 

Based on these conclusions and the fuller discussion of CLASP presented in the final 
report, this report offers four key recommendations for successful continuation of the 
project: 

• CLASP should continue efforts to improve visibility and actively promote 
its successes to secure the support of the community, library administration, 
and city officials. 

• CLASP should develop a full understanding of the needs of schools and 
teachers and ensure that services and resources supplement established 
educational goals. 

• Collaboration must involve librarians working side-by-side with teachers to 
establish program goals that reflect joint planning, mutual benefit, and 
relevance to all audiences. 

• CLASP programming should always create links to books, reading, 
information use, and/ or library services. 



iii 
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CLASP proved to be an important initiative. It showed that libraries can mai- e 
significant contributions to strengthening local communities, promoting education, and 
offering a range of support to the community not otherwise available. The findings and 
lessons resulting from this 20-month evaluation may be useful to funding agencies 
wishing to work with local governments, schools, and libraries developing similar 
projects. 



iv 



CLASP Final Evaluation Report 



IMCS, Inc. 



November 1994 



TABLE OF CONTENTS 



Executive Summary ii 

Introduction 1 

Overview of CLASP 1 

Scope of Evaluation 2 

Methodology 5 

Results/Conclusions 7 

Extensiveness 7 

Satisfaction 10 

Benefit 12 

Effectiveness 13 

Management 15 

Improvement 17 

Critical Success Factors 18 

Innovativeness 19 

Collaboration 21 

Diversity 23 

Other Evaluation-Related Efforts , 25 

Activities and Library Profile Databases 25 

Report on the Use of Management Information Systems 26 

Summer Reading Program Circulation Study 26 

Micro-case Evaluations 27 

Interviews with School Administrators about Collaboration 28 

Memo on Developing Models of CLASP Programming 28 

In-depth Study of School-Library Collaboration 29 

Workshops 29 

Final Conclusions and Recommendations 34 

Moving Forward 37 

References , 39 

Appendix: Methodology 41 



r|c 



V 

8 



CLASP Final Evaluation Report IMCS, Inc. November 1994 



TABLES 

Table 1. Dimensions of Study by Evaluation Period 7 

Table 2. Type of Activity by Number, Attendance, and Location, 

Summer 1994 (Including Summer 1993 Data) 8 

Table 3. Type of Activity by Number and Attendance, Fall 1993 9 

Table 4. Evaluation Activities by Evaluation Period and District 44 

Table 5. Data Collection Instruments by Dimensions of Study, Summer 1993 48 

Table 6. Data Collection Instruments by Dimensions of Study, Fall 1993 49 

Table 7. Data Collection Instruments by Dimensions of Study, Summer 1994 50 



vi 



CLASP Final Evaluation Report 



IMCS, Inc. 



November 1994 



INTRODUCTION 

Overview of CLASP 

In 1991 the DeWitt Wallace-Reader's Digest Fund initiated a three-year pilot program to 
generate local collaborative programming among the New York Public Library (NYPL), 
community schools, and neighborhood organizations. This new library-school 
partnership, called Connecting Libraries and Schools Project (CLASP), 
had three goals: 

1. To establish library use among students at the critical early stages of their 
intellectual development, helping them become better students, 
independent learners and lifelong readers. 

2. To organize teachers, parents, public librarians, school librarians, and school 
administrators in ongoing cooperative activities that encourage the 
productive use of library resources to further children's education and 
family literacy. 

3. To develop a variety of models for cooperative services and institutional 
collaboration that could be replicated in other districts and eventually 
throughout New York City with public funding. 

CLASP operated in three New York City school districts: District 6 in Manhattan, 
District 8 in the Bronx, and District 31 in Staten Island. Twenty-three branch libraries 
were involved: four in Manhattan, eight in the Bronx, and eleven in Staten Island. A 
total of 107 schools from the three districts also participated. CLASP established an 
office in one of the participating branch libraries in each district and employed two 
librarians who worked from each office. The CLASP Director worked from a central 
office and oversaw all program activities. Each CLASP district also had an advisory 
committee comprising school and public librarians, CLASP personnel, teachers, and 
parents. The»e committees met periodically throughout the course of the program to 
discuss the role of CLASP in their districts and the needs of various constituencies. 

The majority of CLASP's resources went into programming for participating branch 
libraries and schools. Programs targeted at young children and their parents, young 
teens, speakers of English as a serond language, disadvantaged people, and others were 
offered not only at the libraries and schools, but also at homeless shelters, summer 
camps, Head Start centers, and other community sites. Typical activities included: 

• Parent Workshops . CLASP librarians read several stories appropriate for 
pre-school children to an audience of parents and their children at a Head 
Start center. They give the parents tips on reading aloud and show the 
parents how to do a simple craft project with their children, based on a 
theme from one of the stories read. They also distribute packets of library 
information. 



0 1 

ERIC 10 

UMM-mmmiM 



CLASP Final Evaluation Report 



IMCS, Inc. 



November 1994 



• Class Visits . CLASP librarians visit a summer school class of elementary 
students. They read some books and discuss the public library. They 
distribute bookmarks, letters for the children's parents, and library card 
registration forms. They also give information packets to the teachers. 

• Craft Programs . A CLASP librarian conducts a session at the library in 
which elementary school students make their own "pop-^p" books. The 
librarian encourages the children to sign up for the summer reading 
program. 

In addition to special programs, CLASP provided material support to participating 
libraries. The program paid for thousands of books to supplement branch collections. 
Through a competitive program, CLASP issued "mini-grants" to teachers in public and 
non-public schools to implement special projects consistent with the goals of CLASP. 

CLASP attempted to link two critical resources — schools and libraries. This type of 
connection has the potential to develop and strengthen: 

• Collaborative programs that enhance educational opportunities for 
students; 

• Family literacy and parent involvement in their child's reading and 
learning; and 

• Local community ties to schools and libraries. 

Successful strategies for school/public library collaboration demonstrated by the 
DeWitt Wallace-Reader's Digest Fund CLASP pilot program may serve as models for 
program replication throughout New York City and other large urban areas. Libraries 
have a major role to play in meeting the national goals for education (Stripling, 1992; 
U.S. Congress, 1994). Projects like CLASP, aimed at coordinating activities of libraries, 
schools, and community agencies, directly address national concerns for improving 
education and meeting the needs of a diverse, multicultural population. 

Scope of Evaluation 

In February 1993, DeWitt Wallace-Reader's Digest Fund hired an evaluation team from 
Information Management Consultant Services, Inc. (IMCS) to conduct an evaluation of 
the final 20 months of the CLASP pilot program. IMCS' original proposal to the Fund 
outlined an empirically-based evaluation using multiple methods and multiple 
comparisons to assess key populations within the context of CLASP's many programs 
and services (Information Management Consultants, Inc., March 23, 1993). The project 
goal was to contribute to both a formative and summative evaluation of the CLASP 
pilot project. 
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While the initial evaluation plan called for four formal studies, DeWitt Wallace-Reader's 
Digest Fund decided early in 1994 to redirect the efforts of the IMCS team. The Fund 
based its decision in part on the 'act that the first three evaluation studies had clearly 
established ways in which CLASP was successful and areas where improvement could 
be realized. During the summer and fall of 1994, therefore, IMCS administered 
activities to help CLASP improve its ability to meet program goals. These activities 
included four workshops for CLASP staff, interviews with school administrators about 
collaboration, an in-depth study of inter-institutional collaboration, and a memo 
describing ways for CLASP staff to approach Goal 3 (development of models). Finally, 
there were some smaller projects, carried out prior to summer 1994, supplementing the 
formal evaluation process. These included the creation of computer databases to record 
CLASP activities, micro-case evaluations (in-depth examination of programs to 
determine critical success factors), and an analysis of the circulation of summer reading 
program materials. 

The evaluation efforts resulted in ten reports submitted to DeWitt Wallace-Reader's 
Digest Fund by IMCS: 

• Report #1: Evaluation Plan (submitted March 23, 1993), describing the 
evaluation strategy for the final 20 months of the CLASP pilot program; 

• Formative Recommendation Regarding CLASP Reporting (submitted June 14, 
1993), discussing the reasons to establish uniform reporting methods and 
the possible uses of activity data to be collected; 

• Report #2: Instruments, Site Selection, Monthly Reports (submitted June 18, 
1993), providing detailed description of the format and nature of existing 
reports provided to the evaluation team, IMCS' organization and use of the 
data, and formative recommendations for CLASP reporting; 

• Report #3: Evaluation of the CLASP Summer 1993 Program (submitted 
September 30, 1993), evaluating the summer 1993 CLASP program activities 
in terms of extensiveness, satisfaction, benefit, effectiveness, improvement, 
and management dimensions of the activities; 

• Report #4: Evaluation of the CLASP Fall 1993 Program (submitted December 
1993), describing the results and analysis of the fall 1993 IMCS data 
collection, which focused primarily on the management of CLASP and 
collaboration between libraries and schools; 

• CLASP Circulation Data — Revised Report [memo] (submitted April 12, 1994), 
presenting the results of an analysis of book circulation to determine 
whether the CLASP summer reading program was successful in promoting 
book use; 
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• CLASP: Collaboration Analysis: Interviews with School Administrators, May 
1994 (submitted June 2, 1994), reporting on interviews conducted with 
senior-level administrative officers from CLASP school districts to 
understand their perspectives toward collaboration; 

• Developing Models of CLASP Programming [memo] (submitted May 31, 1994), 
describing ways to approach the third program goal, to develop a variety of 
models for cooperative services and institutional collaboration; 

• Report #5: Evaluation of the CLASP Summer 1994 Program (submitted August 
. 15, 1994), assessing the degree to which CLASP was able to institutionalize 

recommendations from previous reports, particularly in regard to 
innovativeness, collaboration with the schools, and ability to reach diverse 
audiences; and 

• Report #6: Library Collaboration: A Literature Analysis to Aid the Evaluation of 
CLASP (submitted August 1994), identifying success factors and barriers in 
inter-institutional collaboration and applying those factors in a discussion of 
success and failure in CLASP. 

This, the final report issued by IMCS, reviews the range of evaluation activities carried 
out between February 1993 and October 1994. For the three formal evaluation studies, 
this report describes the approaches taken, sampling and data collection methods, the 
types of variables that were measured, and the methods used to measure those 
variables. The report also describes supplementary activities conducted by IMCS. 
Finally, this report synthesizes the conclusions made throughout the 20-month 
evaluation process and offers key recommendations for the improvement of CLASP. 
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METHODOLOGY 

For all three evaluation studies (summer 1993, fall 1993, and summer 1994), the method 
employed a range of data collection techniques. These included collecting and 
organizing existing CLASP data, and conducting on-site observations, user surveys, 
focus group and individual interviews, and telephone interviews. The initial evaluation 
plan was set forth in Report #2: Evaluation Plan (IMCS, March 23, 1993). The actual 
techniques employed for each study varied, however, according to the particular 
evaluation objectives at the time each study was conducted and the concerns of the 
DeWitt Wallace-Reader's Digest Fund.* 

IMCS carried out the summer 1993 evaluation as specified in the initial evaluation plan. 
A firm conclusion which arose from this first evaluation was that CLASP program 
activities were well-conducted and well -received. With this established, IMCS 
redirected the following two evaluation studies to focus on other issues. The fall 1993 
evaluation focused primarily on the management of CLASP and the collaboration 
between libraries and schools. The summer 1994 evaluation assessed the degree to 
which CLASP was able to institutionalize recommendations from previous reports, 
specifically with respect to innovativeness, collaboration with schools, and ability to 
reach diverse and underserved populations. 

Early in 1994, the Fund, in consultation with IMCS, decided to shift the focus of the 
remaining evaluation efforts to the matter of improvement. CLASP programs had 
already been found to be generally satisfying and beneficial, and they were expected to 
remain so. Instead, the evaluation team would focus on ways to assist CLASP in 
improving its performance for the remaining months of the pilot phase and in 
preparing for continuation beyond the pilot. As a result, on-site interviews with school 
administrators regarding collaboration replaced the complete on-site evaluation 
originally planned for spring 1994. A final on-site data collection for the summer 1994 
program allowed comparison with summer 1993 findings and an assessment of year-to- 
year improvement. IMCS concluded its service to CLASP program by administering 
four workshops for CLASP librarians and administrators and completing an in-depth 
study of inter-institutional collaboration. 

Overall, the CLASP evaluation.£6ncerned itself with three general types of objectives: 

1. Objectives relating public libraries and public schools — especially those 
emphasizing increased and more varied use of public libraries by students 
and teachers; better integration of educational activities with public library 
resources; and improved communication, collaboration, and commitment 
between school professionals and library professionals; 



* IMCS modified the evaluation plan with the advice and consultation of Dr. John Lanigan, Jr., Director 
of Evaluation for the DeWitt Wallace-Reader's Digest Fund. 
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2. Objectives relating children and their families — especially those focusing on 
better use of the public libraries by students and their families; and 

3. Objectives relating public libraries and their communities — especially those 
which affect school-library cooperation. 

The appendix to this report offers a detailed discussion of the evaluation methodology, 
including specific factors examined during each formal evaluation period, the study 
design, and a discussion of sampling, instruments used, and limitations encountered. 
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RESULTS/CONCLUSIONS 

Introduction 

This section contains a general description of the findings of the summer 1993, fall 1993, 
and summer 1994 evaluations, organized by theme or "dimension." For a more 
complete description of the findings for the three formal studies, including quantitative 
data, a sampling of interview responses, evaluators' descriptions of activities observed, 
and testimonials, see the individual evaluation reports (IMCS, September 30, 1993; 
IMCS, December 1993; IMCS, August 1994a). 

The summer 1993 evaluation was a general examination of program quality and 
success. The fall 1993 evaluation focused on management issues. The summer 1994 
evaluation considered whether CLASP had been able to improve in some critical areas 
based on previous recommendations. (For a full description of the way each evaluation 
was conducted, see the appendix to this report.) Table 1 shows the dimensions, or 
themes, which provided the focus for each evaluation. 





Evaluation Period 




Summer 


Fall 


Summer 


Dimensions of Study 


1993 


1993 


1994 


Extensiveness 




• 


• 


Satisfaction 






• 


Benefit 






• 


Effectiveness 








Management 








Improvement 








Critical Success Factors 








Innovativeness 






• 


Collaboration 






• 


Diversity 






• 



Table 1. Dimensions of Study by Evaluation Period 



Extensiveness 

Definition : Number, type, and locations of CLASP activities and attendance at those 
activities. 

Monthly activity logs supplied data on the extensiveness of CLASP programming for 
summer and fall 1993. Individual activity reporting forms supplied data for summer 
1994. (There was some difficulty in comparing program information from one period to 
another because of changes in reporting procedures.) During July 1993, there were 129 
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programs attracting 2,948 participants. Fall 1993 (September through November) 
included 311 programs and 12,992 attendees. For July 1994, records jihow 105 activities 
and 2,795 participants. Tables 2 and 3 describe number and types of activities offered, 
with attendance, for the three evaluation periods. 

The evaluation team expected to see some improvement, based on past evaluation 
recommendations, in programs offered during summer 1994. There were some new 
types of programs in 1994, including hands-on activities and entertainment events. 
Sixteen reported activities for July 1994 were "outsourced," that is, presented by outside 
tai .nt hired by the library. This compares with nine outsourced programs offered 
during July 1993. Attendance of young adults, a target audience, rose slightly from 26 
and in July 1993 to 65 in July 1994. 





# of Activities * 


Attendance * 


Location * 


Type of Activity 


Total 


District 


District 


District 


Total 


District 


District 


District 


Library 


Schools 


Other 






6 


8 


SI 




6 


8 


31 . 








Classes Visited in 


45 


12 


1 


32 


1024 


350 


45 


629 


0 


45 


0 


Schools 


(79) 


(14) 


(22) 


(43) 


(1499) 


(301) 


(423) 


(775) 


(0) 


(79) 


(0) 


Read 


16 


6 


5 


5 


665 


283 


225 


157 


6 


0 


10 


Aloud /Storytelling 


(26) 


(5) 


(3) 


(16) 


(508) 


(270) 


(172) 


(221) 


(10) 


(5) 


(11) 


Parent Workshops 


11 


0 


5 


6 


142 


0 


69 


73 


1 


8 


2 




(9) 


(4) 


(5) 


(4) 


(130) 


(53) 


(58) 


(19) 


(2) 


(5) 


(6) 


Craft/Hands-on 


27 


4 


7 


16 


601 


93 


195 


313 


25 


1 


1 


Activities 


(9) 


(0) 


(0) 


(9) 


(211) 


(0) 


(0) 


(211) 


(9) 


(0) 


(0) 


CLASP at a Community 


3 


1 


1 


1 


248 


80 


83 


85 


0 


0 


3 


Activity 


(2) 


(0) 


(2) 


(0) 


(600) 


(0) 


(600) 


(0) 


(0) 


(0) 


(2) 


Entertainment 


3 


1 


2 


0 


115 


62 


53 


0 


3 


0 


0 




(0) 


(0) 


(0) 


(0) 


(0) 


(0) 


(0) 


(0) 


(0) 


(0) 


(0) 


TOTAL 


105 


24 


21 


60 


2795 


868 


670 


1257 


35 


54 


16 




(129) 


(25) 


(32) 


(72) 


(2948) 


(469) 


(1253) 


(1226) 


(21) 


(89) 


(19) 



* Summer 1993 figures given in parentheses; both periods include the month of July only. 



Table 2. Type of Activity by Number, Attendance, and Location, 
Summer 1994 (Including Summer 1993 Data) 



Although it was not the purpose of the evaluation studies to report complete program 
statistics, it is interesting to note that during its second year, CLASP offered 1,232 
programs to a total user audience of 23,896 children, parents, and teachers (NYPL and 
Branch Libraries, November 15, 1993). The total number of activities carried out during 
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the year was higher than the number of activities originally planned in every category 
except teacher workshops and staff development sessions. CLASP purchased 60,928 
items and allocated them among the 23 participating branches. 



Type of Activity 


# of Activities 


Attendance 


Total 


District 
6 


District 
8 


District 
31 


Total 


District 
6 


District 
8 


District 
31 


Classes Visited in Schools 


141 


35 


79 


27 


4249 


1121 


1897 


1231 


Read Aloud /Storytelling 


38 


23 


5 


10 


1301 


622 


256 


423 


Parent Workshops 


27 


13 


9 


5 


891 


354 


350 


187 


Teacher Workshops/ 
School Librarian Visits 


26 


11 


9 


6 


533 


379 


22 


132 


CLASP at a Community 
Activity 


50 


9 


28 


13 


1900 


157 


1423 


320 


Open School Nights 


21 


2 


18 


1 


3100 


2000 


1000 


100 


Staff Development 


8 


6 


0 


2 


1018 


418 


0 


600 


TOTAL 


311 


99 


148 


64 


12992 

■ 


5051 


4948 


2993 



Table 3. Type of Activity by Number and Attendance, Fall 1993 . 

Programs during the evaluation periods included class visits, parent workshops, 
storytelling /read aloud programs, attendance at community events, craft programs and 
summer reading clubs (for both summer periods), Open School Night displays (during 
the fall), and entertainment events (during summer 1994). 

During the course of the evaluation, it became apparent that NYPL was the first large 
urban library system to implement an outreach program as extensive as CLASP. It was 
difficult, therefore, to compare the nature and extent of CLASP programming to the 
outreach of other urban libraries. In the experience of the evaluation team, however, it 
is clear that CLASP did offer many more programs, especially outside of the library, 
than is typical of similar library systems. CLASP allowed NYPL to serve thousands of 
people who might not otherwise have used the library. 

Conclusions about extensiveness: 



Many of the activities offered were primarily for children and their parents, 
especially during summer 1993. The number of activities varied by district 
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and by type, perhaps reflecting fundamental differences in service 
populations. 

• CLASP appeared to have limited success in reaching older children (i.e., 
those beyond elementary school), at-risk students, and other special 
populations. However, there was a slight increase in young adults 
attending programs during summer 1994. 

• The impact of the CLASP programs appeared to center more on the number 
of activities, rather than the type of such activities, because both CLASP and 
non-CLASP libraries sponsored many of the same type. 

• In comparison to non-CLASP libraries, CLASP libraries did offer more 
options for adults, more contact with community agencies, and a larger 
number of activities. 

• CLASP met or exceeded projected goals in terms of number of activities and 
number of attendees, except in the area of teacaer workshops and staff 
training sessions. The small number of teacher workshops was partly due 
to the delays in beginning the school year, and partly r u e to inadequate 
communication between CLASP and schools. 

• There were fewer activities reported for summer 1994 than for summer 

1993, and attendance was correspondingly less. Inconsistent reporting 
procedures may have resulted in slightly inaccurate program statistics. 

• Interactive programs, involving crafts and other hands-on activities, 
increased in all districts during summer 1994. 

• There was an increase in the number of programs presented by outside 
talent (non-library people hired to present library programs) in summe ' 

1994. In general, the outsourced programs were not designed for CLAe „ 
they were arranged by the NYPL Office of Children's Services and repeated 
throughout the library system with both CLASP and non-CLASP libraries. 

Satisfaction 

Definition : Participants' reactions to CLASP activities, including the degree to which 
they believed CLASP activities met their needs. 

User surveys and telephone interviews conducted during the summer 1993 evaluation 
included questions about satisfaction. The evaluation team distributed surveys at 52 
activities representing all three districts, which elicited the responses of 751 patrons (706 
youth and 59 adults). Results indicated that respondents were overwhelmingly 
satisfied with the activities or events in which they participated. Among children, 98% 
liked the activity, 94% felt it was interesting, 92% said they would like to participate 
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again, and 88% believed their friends would like it. Of the adult respondents, 90% felt 
the activity was enjoyable, 85% found it to be very interesting, 87% showed interest in 
coming to similar activities, 85% thought the activity had taken the appropriate amount 
of time, and 95% felt the appropriate amount of information had been presented. 
CLASP librarians interviewed by telephone expressed the opinion that CLASP had a 
positive effect on their level of satisfaction with their ability to perform professionally. 
Librarians also appreciated the additional resources purchased by CLASP. One stated, 
"The librarian doesn't have to tell them there's nothing.. .they walk out with something 
in their hands." 



Because a high level of satisfaction had been so clearly established in the summer of 
1993, this theme was not studied in the fall and was looked at as a secondary concern 
only during the summer 1994 evaluation. Observation of activities in summer 1994 
showed that participants were interested and involved, and that they expressed a 
similar level of satisfaction with both "non-capacity" and "capacity"* activities. In 
interviews, parents stated that they believed their children liked these activities, that 
their children were learning, and that the activities promoted creativity and library 
interest. Interviews with outside talent (people hired to present library programs) 
indicated that they believed participants were satisfied with the activities. 



Conclusions about satisfaction: 



Teachers, librarians, parents, and participants seemed highly satisfied with 
the CLASP activities in all three districts for both summer 1993 and summer 
1994. This was true for every type of respondent and all facets of 
satisfaction considered. 



Adults appreciated CLASP services and the increased resources available as 
a result of the program. They expressed the wish that access to these 
programs and resources be improved so that more families could benefit 
from CLASP. 



The great majority of participants judged the content of CLASP activities to 
be appropriate and pleasing, despite the difficult physical conditions of 
some of the presentation sites. 

Branch library staff were highly satisfied with the enhanced collections 
made possible by CLASP, because the additional materials helped them fill 
chronic demands. 



* "Capacity" refers to the activities chosen for observation in summer 1994 because the CLASP Director felt they best 
demonstrated CLASP's capacity to implement improvements based on past suggestions. "Non-capacity" refers to the 
"ordinary" activities selected and observed by the evaluation team during summer 1994. 
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Benefit 

Definition : The way(s) in which CLASP activities contributed to the quality of life for 
participants. 

As with satisfaction, the IMCS team assessed benefit most closely during the summer 
1993 evaluation, when it was found to be generally positive. There are some particular 
difficulties in judging benefit, which can be either immediate or delayed, direct or 
indirect. These difficulties are particularly at issue in assessing the benefits of library 
programs for children. The tangible benefits of using the library, enjoying book talks, 
and starting to read are not likely to become manifest immediately. Similarly, if library 
programs for parents are effective, the benefits and changes in attitude will accrue to 
their children, but usually after some time has passed. Therefore, benefits noted here 
may be different from what they would be if the questions were asked in the future. 

Of 706 children who participated in surveys, 91% said they had learned something 
during the CLASP activity, and 85% found the activity useful or helpful. Of the 59 adult 
respondents, 88% thought the activity had been useful or helpful, and 95% believed the 
amount of information provided at their activity was appropriate. Of eight librarians 
interviewed by telephone, six believed CLASP activities were moderately to very 
educational and useful for their patrons. Two others believed the library's ability to 
purchase more materials, with CLASP funding, was very important for the quality of 
public service. Observations of CLASP activities also showed positive evidence of 
benefit to users, such as immediate registration for library cards and selection of books 
to take home. Interviewees agreed that CLASP had a positive impact on students and 
parents, especially in introducing people to libraries, teaching library use and 
stimulating interest in using libraries, providing useful information services, and 
reinforcing the importance of reading for children and adults. 

In general, respondents saw the summer 1994 capacity activities (i.e. "improved" 
activities selected for observation by the CLASP Director) as helpful and useful for a 
variety of reasons from assisting children in expressing their inner feelings., to helping 
them when doing research activities in school, to having a useful product to take home. 
They saw a film-making project with special education students as an opportunity for 
students to be involved in something that was motivating and allowed them to use 
artistic and creative talent in new ways, as well as to practice communication and 
conceptual skills. Non-capacity activities (i.e. "ordinary" activities similar to those 
offered in the past) evoked similar responses, with parents noting that some program 
activities were related to things their children did at home, and that the programs 
helped their children become comfortable with the library. Following a storytelling 
program, one child said, "It helps you learn to do storytelling and makes you use your 
imagination... you get a picture inside your mind." (He had used the skills learned in the 
library's storytelling programs to win a storytelling contest for children.) 
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Conclusions about benefit: 

• In both July 1993 and July 1994, youth and adult participants found CLASP 
activities largely beneficial and were able to name ways in which the 
experience gained at a CLASP program might carry over into their home, 
school, or work life. 

• Both teachers and CLASP staff felt that activities benefited the participants 
in terms of enjoyment and information in about equal measure. 

• CLASP participants reported at least some short-term benefits, although 
their satisfaction with the overall CLASP experience was somewhat greater 
than their perception of benefits received. 

• Librarians considered CLASP's ability to provide additional resources and 
support personnel to be among the most important benefits. 

• Supporting reading for pleasure and providing access to a larger stock of 
materials appear to have been key benefits for students and parents alike. 
The programs also gave parents new skills and ideas for encouraging 
reading and language development in their children. 

• Many parents and teachers showed an immediate interest in follow , t g up 
upon their CLASP experience, suggesting that the programs were beneficial 
to them (e.g. teachers immediately scheduled class library visits, parents 
borrowed books the same day, etc.). 

• Despite increased frequency in CLASP activities, there appeared to be no 
difference between the impact of CLASP and non-CLASP activities on their 
respective audiences, as judged by librarians in different institutions. 

Effectiveness 

Definition : Extent to which the activity achieved the goals of CLASP. 

There was considerable evidence of success in terms of Goal 1 (establish library use 
among children) during summer 1993. The high levels of perceived satisfaction and 
benefit attested to the fact that CLASP had a positive effect on users. Observations 
revealed that programs did encourage student library use, by familiarizing children 
with the library, giving them library card applications, exciting them about reading, 
introducing them to storytelling, etc. One principal commented, "The kids feel like they 
[CLASP staff] had to shut down everything to come and visit them, so it makes them 
feel special." User surveys showed that people considered the library to be more 
accessible and attractive to them after attending CLASP activities, with a large majority 
of adults and youth indicating interest in attending more programs. 
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The fall 1993 evaluation, also found CLASP to be highly successful in terms of Goal 1. 
Advisory committees in all three districts mentioned that CLASP brought new children 
and adult patrons into the library, that the added materials were crucial to the ability to 
serve them, that parent outreach was important and effective, and that the summer 
reading lists were a successful means of promoting reading for pleasure. Adults were 
enthusiastic after learning what resources were available through the library and 
expressed an interest in returning. Children and teachers enjoyed the storytelling 
sessions very much. During the September through November period, 2,856 library 
cards were issued to attendees of CLASP programs. 

In terms of Goal 2 (organize librarians, parents, and school personnel in cooperative 
activities), the summer 1993 evaluation found that CLASP succeeded in part. Branch 
librarians believed the cooperation and support they received from CLASP personnel 
were significant, although programs presented in libraries sometimes showed 
insufficient coordination between CLASP and library staff. The level of cooperation 
between libraries and schools varied. There was little evidence of jointly planned and 
executed programs. Some teachers did feel that the program contributed to 
coordination between school, home, and library by encouraging student use of the 
library, and they appreciated the help of CLASP in terms of resource selection and 
program ideas. 

The fact that many programs took place in schools contributed to Goal 2, because 
CLASP personnel were brought into contact with teachers and principals. Some 
librarians, however, felt that teachers questioned their authority. Finally, there was 
little direct evidence of CLASP's ability to make parents effective contributors to their 
children's education, but the favorable reaction of parents to CLASP programs 
suggested a positive effect. 

The fall 1993 study found that CLASP satisfied Goal 2 to the extent that there were 
many library- ponsored activities in which the schools took some part. CLASP/NYPL 
management believed CLASP had demonstrated the importance of community 
outreach and had succeeded in raising schools' and the larger community's awareness 
of the library. Also, there was considerable evidence t.iat parents and the community 
had an increased awareness of library resources as a result of CLASP. School program 
administrators believed CLASP enhanced the school program and had a positive effect 
on children and parents although, again, there was little in the way of cooperative 
programming between schools and libraries. 

Conclusions about effectiveness: 

• There was evidence that CLASP had a positive impact on family literacy for 
those attending CLASP activities. 

• The quality of CLASP staff and support was high. 
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• CLASP staff were able to reach some community members who were not 
users of the library. In addition, librarians reported more children, teachers, 
and parents using the library. 

• Because of CLASP, librarians in non-CLASP libraries demonstrated an 
increased interest in providing outreach programs and services. 

• There did not appear to be much of a difference between CLASP and non- 
CLASP libraries in termb of how patrons reacted to library activities. 

• Although branch libraries sponsored many of the same types of activities, 
CLASP appeared to approach them in a more systematic and aggressive 
fashion. 

• While CLASP apparently made good progress in contacting students 
through activities, the teachers were generally unaware of the project. 
Word of the program may not have filtered down to teachers from the 
administrative level. Once having participated in CLASP, however, 
teachers were uniformly enthusiastic about the project. 

• Many of the schools and community agencies scheduling parent workshops 
during the first year (at the invitation of CLASP) requested repeat 
performances for new clientele. 

• Branch staff exhibited a high degree of support for CLASP despite their lack 
of personal experience with program activities. They believed that NYPL 
was engaged and interested in CLASP and that CLASP was making 
significant contributions to institutional priorities. 

• There appeared to be some interaction between teachers and CLASP 
librarians for arranging and implementing activities. There was little 
evidence, however, that teachers were regularly involved in program design, 
either individually or in some other forum. 

• Some decisions related to curriculum continued to be made autonomously 
by CLASP and library personnel with little or no consultation with students, 
teachers, or school administrators. 

Management 

Definition : Extent to which CLASP was well managed and well marketed. This theme 
included the decision making process, the organizational structure, the delineated 
authority and responsibilities within the organization, the cost-effectiveness of the 
program, and documentation supporting the structure. 
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In terms of management, the summer 1993 evaluation focused on av, areness of CLASP 
by librarians, teachers, parents, and the public, as well as librarians' beliefs about the 
value of the project. Interviews revealed that these people were not consistently aware 
of CLASP. While librarians in both CLASP and control libraries said they knew about 
CLASP, school teachers and principals often had not heard of the program. Members of 
the public generally did not know about CLASP either, although they sometimes 
recognized CLASP librarians from activities. Among parents ard children, some knew 
of CLASP activities through publicity at the schools, library bulletin boards, and local 
newspapers, but most were unaware of CLASP. 

Regarding the value of the program, librarians believed that the sheer increase in 
resources made available through CLASP funding — multiple copies of books to support 
summer reading lists and school assignments, as well as reference books, magazine 
subscriptions, software, and equipment — was one of the most valuable benefits of the 
program. Librarians also appreciated the personnel support provided by CLASP. They 
noted that CLASP staff raised awareness about outreach, supplied programming ideas, 
helped ease the workload at the branches, and helped them meet their demand for 
activities, especially class visits. 

The fall 1993 evaluation focused largely on the opinions of CLASP/NYPL management,, 
school staff, and advisory board members. Management believed that CLASP provided 
a good model of cooperative service for the library system in general, as well as a 
variety of cooperative models for activities and projects. School staff said they faced 
barriers such as lack of flexible scheduling, lack of professional librarians, and 
inadequate resources. Members of the advisory committees believed CLASP money 
was well spent: 73% of respondents said that the average activity was worth $1000, 91% 
thought the activities were worth $500, and 100% thought the activities were worth at 
least $100 (23 respondents total). School program administrators noted that no program 
literature was available other than flyer? for individual activities. Parents who were 
interviewed did not realize CLASP was distinct from other library activities, and school 
staff in two districts did not believe their administrators knew anything about CLASP. 

Conclusions about management: 

• Although the quality of general activity planning was solid, specific 
planning was often done at the last minute, hindering coordination, 
publicity, creativity, and scheduling. 

• There was little follow-up or evaluation of specific programs to determine 
key success factors. 

• Branch librarians saw CLASP as a way to extend their materials and 
personnel resources. 
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• There were instances of lack of awareness of CLASP, i.e. that specific 
activities were CLASP-sponsored. CLASP was better known among library 
and school professionals than among parents and children. 

• There was little use of computer or developing information technologies in 
planning, presenting, or managing programs or activities. 

• Many respondents identified CLASP librarians with NYPL rather than with 
CLASP. For the most part, the people served by CLASP were unaware that 
CLASP was the sponsor of these special programs, activities, and resources. 

• CLASP began to demonstrate some of the potential of what the public 
librarian can do in terms of outreach and working with the schools. The 
successes of CLASP began to generate demand for comparable services and 
materials in non-CLASP libraries. 

Improvement 

Definition : Ways to maintain, correct, or improve CLASP procedures or activities, from 
respondents' point of view. (During summer 1994 this was studied in terms of three other 
themes, innovativeness, collaboration, and diversity, which are described below.) 

Most suggestions for improvement during summer 1993 were minor. User comments 
included: reduce the length of a preschool story hour and select books of a higher level, 
publicize summer programs more widely and hold them earlier in the summer, provide 
more time for a craft project, make storytelling/booktalking sessions more enticing, etc. 
Micro-case studies (in-depth examination of programs to reveal factors contributing to 
success or failure) revealed that communication, cooperation, and coordination were the 
most critical factors in successful CLASP programming. Second in importance were 
resources to support the activities (e.g. refreshments, craft supplies, etc.). In general, 
most of the summer 1993 cases showed the potential to be successful, especially if 
CLASP/NYPL were to focus on improving the critical success factors identified in each 
case. 

For the fall 1993 evaluation, &l. "isory committee members were eager to expand current 
services to other areas, add new client groups or services to current CLASP activities, 
and, above all, to sustain CLASP in its current form. All expressed concern over the 
status of continuation funding, as well as the importance of increasing program 
awareness among high level school administrators and within the city political arena. 
Interviewees also noted that public awareness and program publicity needed 
improvement. A school librarian in District 6 stated that students would be more likely 
to go to the library from school if safety were improved (e.g. by bus transportation). 
Students thought class visits could be improved by using more CDs and videos, giving 
participants books to keep, showing them how to use computers, and providing 
resource lists. A school staff member thought the library ought to regularly share 
community information with teachers. 
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Conclusions about improvement: 

• Adequate communication, cooperation, and coordination are essential to 
improving CLASP activities. 

• Libraries and schools must increase communication to address 
curricular/ instructional needs. Schools did not perceive themselves as full 
partners in collaboration; CLASP librarians had difficulty in obtaining 
information about curricular needs of schools. 

• CLASP staff in the three districts wanted/needed more frequent 
opportunities to work together. 

• CLASP must increase program publicity and boost public awareness. 

• Inexpensive resources (e.g. refreshments, paper supplies, pamphlets) were 
often critical to successful implementation of activities. 

Critical Success Factors 

Definition . Extent to which the presence/ absence of a factor determined the 
success/ failure of an activity, from the respondents ' point of view. 

During interviews in the fall 1993 evaluation, CLASP/NYPL management named the 
following factors as critical to CLASP program success: 

• Librarians must have excellent communication skills to work with students, 
parents, and teachers. 

• Librarians must speak Spanish in certain districts. 

• Libraries must be able to meet increasing demands for materials in a 
growing and diverse user community. 

• Forums must be established for schools and libraries to communicate. 

• CLASP must be perceived as valuable to the schools in order to gain their 
commitment. 

• Models to replicate CLASP programming must allow flexibility to adapt to 
local circumstances. 

• Students should enjoy CLASP library visits. 
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• CLASP should aim to register every child visiting the library for a library 
card, which would give CLASP a powerful statistic to use for future 
funding. 

School program administrators stressed the importance of good communication 
between CLASP and the school program director, continuity of events, the ability to 
create ties with schools and agencies to involve parents and children, and the need for 
dynamic CLASP librarians. 

Conclusions about critical success factors: 

• CLASP librarians must possess excellent communication and motivation 
skills. 

• Advisory committees need to serve as forums for decision-making, 
suggesting new and innovative programming ideas rather than validating 
existing programs and services. 

• School personnel (principal and teachers) need to be directly involved in 
decisions about program planning and implementation. 

Innovativeness 

Definition : Extent to which CLASP instituted new approaches or new programs. 

The fall 1993 evaluation sought the opinions of teachers, librarians, and others about 
program innovativeness. School program administrators believed CLASP was 
innovative in general, and very innovative when it used special programs to bring 
parents and children into the library. Some respondents considered outreach (offering 
programs outside the library) to be innovative. Teachers believed CLASP was 
innovative in general and had stimulated children's interest in the library. 

Others noted that the CLASP mini-grant program* had spurred creativity, allowing the 
schools to offer something different. In contrast to the opinions expressed in interviews, 
observation of some class visits by the evaluation team revealed that librarians were 
using standard, not innovative, approaches, and that these appeared to be of limited 
interest to the youth. 

The summer 1994 evaluation examined innovativeness as one of three specific areas of 
program improvement. (The others were collaboration and diversity.) Capacity 
activities (i.e. "improved" activities specifically identified for observation by the CLASP 
Director) showed some evidence of innovation: some of the presenters made reference 



* CLASP awarded "mini-grants" to teachers in public and non-public schools to implement special projects 
consistent with the goals of CLASP. 
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to school curriculum and consciously included activity skills related to the curriculum 
of early grades. Librarians in District 6 believed a program of Dominican music was the 
first event aimed at Spanish-speaking adults to be offered at their library. Parent 
interviews revealed that they thought a hands-on craft activity was innovative, as the 
programs they had seen in the past usually involved reading or singing only. The non- 
capacity activities (i.e. "ordinary" activities similar to those offered in the past) showed 
less evidence of innovation, although some programs were new and different in that 
mey involved hands-on, participatory activities. 

CLASP programs made few connections to educational or curricular goals, and CLASP 
librarians were not always able to help the children see those connections by stating or 
demonstrating them. Presenters often missed opportunities to extend the subject 
matter of the activities to stimulate children's imagination or to make connections to 
literature or library use. There was not always a CLASP representative at activities 
given by outside talent (i.e. non-library people hired to present library programs), and 
often the presenters made no attempt to identify CLASP or make a connection between 
the activity and library services and/ or resources. 

Capacity and non-capacity activities (designated as such during the summer 1994 
evaluation to determine whether improvements had been made after past evaluations) 
showed little difference in innovativeness overall. One notable improvement was 
making many of the activities for young children more interactive. 

Conclusions about innovativeness: 

• Some programs were the first of their kind and brought new experiences to 
the participants. 

• While the majority of the activities were book-oriented to some degree, 
presenters did not always make a strong connection between the 
activities presented and reading and literature appreciation or library 
resources. 

• Through the schools, CLASP initiated new, innovative outreach programs 
for parents, such as parent workshops and Open School Night 
presentations. 

• Teacher workshops and CLASP librarian visits to classrooms were unique 
to CLASP. 

• Respondents perceived innovativeness as tailoring programs and services to 
meet the specific needs of a given community. Therefore, what was 
considered innovative in one community may not have been in another. 
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• The CLASP mini-grant program resulted in some highly creative and 
unique programs such as the "Let's Communicate" Symposium, a series of 
special activities culminating with an all-school assembly and visits to 
classrooms by various high-profile members of the community, including 
CLASP librarians. 

• When outside talent (i.e. non-library people) presented library programs, 
there was insufficient connection to CLASP, to CLASP staff, and to library or 
project goals. 

• Activities were often input-oriented (providing participants with resources) 
rather than process-oriented (stimulating participants to think and learn). 

• Interactive programs for young children did not always make clear 
connections to library resources, reading, or books. 

• Aside from including more interactive activities for children, capacity 
programs were not clearly more innovative than non-capacity programs. 

Collaboration 

Definition : Extent to which CLASP staff worked together effectively with teachers, 
administrators, parents, and community organizations to further the goals of the 
project. 

During the literature analysis conducted in the summer of 1994, researchers discovered 
that many authors distinguish between collaboration and cooperation (see p. 29; also 
IMCS, August 1994b, p. 4-6). According to this scheme, collaboration is the more 
fundamental relationship, involving mutual planning, a goal that neither institution 
could achieve alone, joint acquisition of funds, formal channels of communication, 
shared management and control, and equivalent contributions of time and resources. In 
contrast, cooperation is a less demanding relationship, involving short-term work with 
limited goals, one organization providing leadership and obtaining funds, occasional 
sharing of information, and benefits accruing primarily to one institution. Those who 
planned and implemented CLASx 1 did not clearly define the nature of the intended 
relationship between schools and libraries or consistently label it as either collaboration 
or cooperation . The opinions of people asked about collaboration during the fall 1993 
and summer 1994 evaluations, therefore, reflect various assumptions about the meaning 
of the term and the type of school-library partnership CLASP was to develop. 

CLASP/NYPL managers surveyed in fall 1993 rated collaboration as "adequate" to 
"successful." However, the collaboration was one-sided with schools passively 
receiving CLASP services. Advisory committee members believed that CLASP had 
improved rapport with teachers, that there was more library use by teachers, and that 
public librarians in general were learning more about how to work with schools. School 
program administrators felt that CLASP formed an important link between schools and 
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libraries; other school staff noted that schools were not involved in planning or goal 
setting for CLASP, but that CLASP did offer the assistance of professionals. A 
comparative interview between CLASP libraries and non-CLASP libraries in the New 
York Public Library system showed that CLASP institutions were engaging in more 
joint planning than non-CLASP institutions, but that cooperation was most evident in 
the delivery of activities. Furthermore, there was little regular communication between 
teachers and librarians about resources and curriculum. 



The summer 1994 evaluation examined collaboration as one of three specific areas of 
program improvement. (The others were innovativeness and diversity.) The capacity 
activities (i.e. "improved" activities selected for observation by the CLASP Director) did 
show some evidence of collaboration. A film-making project, for example, involved 
ongoing work between a branch librarian and a classroom teacher and collaboration in 
planning, implementation, and follow up. In other activities observed, however, 
collaboration between schools and libraries involved joint scheduling only. Interviews 
with outside talent (i.e. non-library people hired to present library programs) revealed 
that they had not been told by the library that there were any specific goals or needs and 
therefore had not tailored their presentations to library concerns. Similarly, the non- 
capacity activities (i.e. "ordinary" activities similar to those offered in the past) showed 
little or no collaboration. Class and agency visits appear to have been coordinated with 
administrators with minimal involvement of teachers. In some programs using outside 
talent, the CLASP librarian was strictly an observer or was not present. 



Conclusions about collaboration: 



Both libraries and schools perceived an increase in school-library 
collaboration on the activity level as a result of CLASP. Increased 
interaction created more extensive relationships between CLASP librarians 
and school personnel. 

Collaboration appeared to emphasize joint scheduling rather than planning 
and goal setting, design, delivery, and evaluation of activities. 
Collaboration was largely one-sided; either CLASP provided the services 
and the schools were passive recipients, or the schools requested activities 
and directed CLASP's involvement. 



CLASP enabled more collaborative or coordinated efforts to occur, but the 
success of those efforts and the nature of those efforts were not much 
different from what non-CLASP libraries reported they were doing with the 
schools. The presence of advisory committees was one clear difference 
between CLASP and ncn-CLASP collaboration with the schools. 

Within the schools, there were institutional barriers that affected 
collaboration efforis, such as inequitable distribution of resources among 
schools, failure of school libraries to receive state allotments for book 
purchases, and limited rewards and incentives for participation in CLASP. 
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• Some CLASP librarians were instrumental in helping teachers working as 
school librarians (who generally had no formal library training) to develop 
library collections and services. 

• The advisory committees were well attended and perceived as the only 
formalized collaboration process between libraries and schools. However, 
they were dominated in number by CLASP and NYPL staff. 

• CLASP staff made strong and consistent efforts to cooperate with 
community agencies to deliver services to disadvantaged populations at 
community sites. 

• Although one program (film-making) reflected a relatively high degree of 
cooperative planning and implementation, there were no clear differences 
between capacity and non-capacity activities in terms of school-library 
collaboration. 

Diversity 

Definition : Extent to which CLASP programs drew audiences representative of the 
service districts, particularly underserved populations including older children (i.e. 
those beyond elementary school), at-risk students, people of various ethnic backgrounds, 
and people of different ages. (This theme was examined during summer 1994 only, and 
considered under the scope of program improvement.) 

The summer 1994 evaluation examined diversity as one of three specific areas of 
program improvement. (The others were innovativeness and collaboration.) Of the 
capacity activities (i.e. "improved" activities selected for observation by the CLASP 
Director), some reached diverse audiences while others did not. Parents felt the library 
was successful in reaching out to people in the community who already frequented the 
library, but believed that non-users would have no way of learning about the programs 
because they had only seen the programs advertised within the library. A children's 
librarian noted that different types of activities attracted different groups of the targeted 
children. The programs directed at teens appear to have attracted the most diverse 
audiences in terms of ethnic makeup, age, and gender. Some at-risk and special 
education teens also participated in these activities. Teen attendance was slightly higher 
in summer 1994 than in summer 1993. 

In non-capacity activities (i.e. "ordinary" activities similar to those offered in the past), 
the level of audience diversity was varied. A branch librarian in Staten Island said that 
her library was attracting more of the economically disadvantaged members of the 
community. Staff placed advertisements for activities primarily at branch libraries, 
where they would not be seen by non-users. Adults in Staten Island suggested offering 
activities on evenings or weekends in order to accommodate families with working 
parents. Parents also expressed a belief that library resources were better in the more 
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affluent districts and areas within districts; however, some believed the situation had 
improved since the advent of CLASP. One remarked, "we were ignored before, but 
now they're really reaching to everyone." Some parents felt that one of the most 
successful strategies CLASP used to reach out to the diverse groups within the 
community, especially non-English speaking people, was to visit the schools to 
introduce the children to the library and register them for cards. 

Conclusions about diversity: 

• CLASP had a positive impact on the equitable availability of resources to all 
populations served in CLASP districts. 

• One of the most successful and well-received efforts to reach diverse 
audiences, particularly non-English speaking people, was visiting schools 
and registering children for library cards. 

• CLASP reached out to Spanish-speaking communities by offering a 
significant number of programs in Spanish. Some programs, such as a 
Dominican music event, succeeded in attracting many non-users. 

• Although some activities drew ethnically diverse audiences, others did not 
but might have done so with better promotion. Efforts to advertise 
activities in places where potential new audiences might be reached were 
inadequate. 

• Overall, capacity programs (i.e. those "improved" activities selected for 
observation by the CLASP Director) were no more successful in reaching 
diverse audiences than non-capacity programs. Issues such as the need for 
adequate advertisement applied to both types of programs. 
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OTHER EVALUATION-RELATED EFFORTS 



During the course of the evaluation period, IMCS conducted several activities 
supplementing the formal, on-site evaluation studies or otherwise related to the 
evaluation and improvement of CLASP. Several of these efforts were carried out 
during the summer and fall of 1994, after the DeWitt Wallace-Reader's Digest Fund 
decision that the evaluation effort should shift to include more direct interventions with 
CLASP/NYPL and provide the Fund with information from the CLASP pilot study that 
might be useful with future grantees. The additional evaluation-related activities 
included: 



Databases with activity and institutional information; 
A report on the improvement of reporting procedures; 
An analysis of circulation data for summer 1993; 
Micro-case evaluations; 

A memo describing ways to address Goal #3 (developing models of CLASP 
programming); 

Interviews with school administrators about collaboration; 
An in-depth study of school-library collaboration; and 
A series of workshops for CLASP/NYPL. 



These activities assisted the evaluation team in completing and extending the original 
evaluation objectives. 



Activities and Library Profile Databases 



The IMCS team created two computer databases in the spring of 1993 to support their 
evaluation and program improvement efforts. One database was for activity 
information, and one was for institutional information. The rationale for developing the 
databases and sample reports from the databases are included in Report #2: Instruments, 
Site Selection, Monthly Reports (IMCS, June 18, 1993). The purpose of the databases was 
to provide baseline data, help create reports based on specific variables, and facilitate 
comparative analysis. 

Activities Database . The activities database contained details on specific CLASP actions 
from the beginning of the project through April 1993. It recorded data obtained from 
existing monthly reports. To account for variations in the way such data was 
previously reported, IMCS chose the activity as the primary unit for organizing the 
data. The database comprised one record for each activity conducted by CLASP. 
Separate fields for audience, location, staff, etc. contained data specific to each activity 
record. 



Library Profile Database . The institutional profiles database recorded data about all 
libraries involved with CLASP, as well as control libraries to be used in evaluation. The 
database included one record for each library, with data such as district, address, 
population breakdown, circulation, and budget entered in separate fields. 
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After inputting the initial batch of data into each database, IMCS gave the project to the 
CLASP Director and trained him in how to input data for further record keeping and 
analysis. 

Report on the Improvement of Reporting Procedures 

A document issued on June 14, 1993, Formative Recommendation Regarding CLASP 
Reporting (IMCS, June 14, 1993), covered the use of data reports provided by CLASP and 
NYPL. The report described the nature and format of the existing data, how they were 
being organized and used, and ways in which CLASP reporting could be improved. 
The issues identified in the report were those which drove the creation of the activities 
and institution databases (see above). The new databases were designed to make it 
possible to compare data from a range of sources; create reports that address specific 
questions; generate baseline data profiles of comparison to data collection undertaken 
by the evaluation team; and report data in a number of formats not presently available. 

This memo suggested that a worksheet be designed that would allow all CLASP sites to 
report uniformly. The worksheet should provide room for narrative comments, 
including anecdotes, assessment of needs, problem identification, personal reflections, 
and self-evaluations. 

Summer Reading Program Circulation Study 

IMCS conducted a circulation analysis in summer 1993 to determine whether CLASP 
was effective in promoting book use through its summer reading programs (IMCS, 
April 12, 1994). The study included both CLASP and non-CLASP branches so that 
statistics could be compared. For the titles being studied, evaluators determined a 
utilization rate ; this took into effect both circulation and the number of copies of each 
title available. Evaluators then compared circulation statistics for late spring, before the 
summer program had begun, and early fall, after the summer program had ended and 
all circulation transactions were complete. Because of limitations of the NYPL 
circulation system, this calculation proved to be extremely labor intensive. 
Consequently, IMCS chose a subset of the reading list titles to study. This subset was 
Spanish language books, selected because it related to a target population and because it 
included all grade levels served by the summer reading program. 

To complete the study, IMCS identified a similar library program at another library 
system (Onondaga County Public Library, Syracuse, New York), and determined the 
utilization rate for its Spanish language materials for comparison to CLASP statistics. 
IMCS considered Onondaga County Public Library to be a suitable institution for 
comparison to CLASP/NYPL because the two library systems shared several important 
variables: (1) both had special funding to increase circulation of foreign language 
materials; (2) both had an established program to carry out this objective; and (3) both 
were being compared after a similar period of time (about one year). 
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Findings from the study included: 

• Spanish language books advertised as part of the CLASP summer reading 
program circulated more than the same titles in the non-CLASP libraries. 

• This increase in circulation could not be accounted for by the fact that the 
CLASP libraries had more copies of these titles than the other branches. 

• The Spanish language books circulated more times per copy than did 
Spanish books at another library (Onondaga County Public Library), even 
though the other library's circulation was based on a full year. 

• The various promotional efforts of CLASP appeared to contribute to 
increases in the use of Spanish language books. 

Micro-Case Evaluations 

CLASP conducted several micro-case evaluations in conjunction with the summer 1993 
and fall 1993 evaluations. These evaluations were in-depth examinations of CLASP 
programs determined to be particularly successful or in need of improvement. By 
interviewing program participants and those planning and conducting the programs, 
evaluators were able to determine the factors leading to the success (or preventing 
success, in the case of programs needing improvement). The activities selected for 
micro-evaluation can serve as "models" for other libraries and districts. 

The summer 1993 evaluation included 16 micro-case evaluations. Programs examined 
were a workshop for day care center teachers, a kindergarten class visit, a treasure hunt 
activity (funded by a CLASP mini-grant), a Spanish language parents' workshop, a 
teachers' workshop series, and others. Seven micro-case evaluations performed in fall 
1993 included a family literacy conference for community agencies, a library orientation, 
"superstart" preschool parents workshop, and others. 

The micro-case studies resulted in the identification of critical success factors. While 
many were specific to the types of programs examined, some were more general and 
may be applied to other programs. Following is a sample of the critical success factors 
elicited through the micro-cases: 

• More instruction in Spanish for staff and adequate amounts of Spanish 
language materials are needed to meet the new demand [created by a 
bilingual parents workshop]. 

• A core group of committed planners must cooperate, coordinate, and 
communicate freely and frequently [for a family literacy conference]. 

• Presenters must know the library needs and interests of elementary school 
children, their teachers, and their parents [for a library orientation]. 
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• School and library administrators must cooperate and ensure support in 
program planning, promotion, and attendance [for a teachers' workshop 
series]. 

Complete reports of all micro-case evaluations are appended to the summer 1993 and 
fall 1993 reports (IMCS, September 30, 1993; IMCS, December 1993). 

Interviews with School Administrators about Collaboration 

In the spring of 1994, a team from IMCS conducted in-person interviews with eight 
senior-level administrative officers from the three CLASP school districts and three 
administrators at the New York City Board of Education. The team's intention was to 
understand the perspectives and attitudes of top school administrators toward 
collaborative efforts between schools and outside organizations in general and in the 
CLASP project specifically. IMCS presented the findings in a report, CLASP: 
Collaboration Analysis: Interviews with School Administrators (IMCS, May 1994), and also 
used the findings when conducting the in-depth study of school-library collaboration 
(described below). 

Administrators considered projects with which they were familiar (other than CLASP) 
that involved collaboration between one or more schools and an external agency or 
organization. They named buy-in by district level administrators and programs aimed 
at meeting the instructional and curricular needs of the schools as the most important 
factors for successful collaboration. In addition, the group identified the following as 
necessary to successful collaboration: strong leadership; the setting of realistic goals; 
joint planning and decision making; adequate preparation and training; and sufficient 
resources. 

The interviewees identified several barriers, including lack of planning time, lack of 
training, inadequate hours and staffing of the public library, physical barriers (e.g. 
distance, incompatible hours), bureaucratic barriers, lack of involvement of t> >p 
administrators, "turf" issues, and not enough activities taking place in the schools. 

Memo on Developing Models of CLASP Programming 

IMCS submitted a memo to the CLASP Director (IMCS, 1 iay 31, 1994) to provide some 
assistance in beginning to carry out Goal 3: To develop a variety of models for 
cooperative services and institutional collaboration that could be replicated in other 
districts and eventually throughout New York City with public funding. (Previous 
evaluation reports had noted that little progress had been made in achieving Goal 3, 
possibly due to the lack of a unified effort to define what such models might look like.) 

The memo identified three types of models, one at the political /institutional level, one 
at the school/ community level, and one at the program /activity level. It recommended 
that the type of model be chosen according to the intended purpose. Throughout the 
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modeling process, the memo recommends: (a) selecting programs appropriate to local 
needs and interests; (b) adapting programs to meet local conditions; and (c) carrying out 
programs so that they are bound to succeed. In addition it is important to consider: (a) 
the local political context as it affects the community at large, the public library and the 
schools; (b) the make-up of the community in terms of ethnicity, primary language, 
cultural values, etc.; and (c) the status of local schools in terms of resources and 
professional staff. 

In-depth Study of School-Library Collaboration 

Researchers undertook a literature search and analysis to help identify factors that may 
have contributed to successes and failures in CLASP school-library collaboration. The 
analysis also identified models of inter-organizational collaboration, and identified 
factors critical to successful collaboration between public sector institutions. Report #6: 
Library Collaboration: A Literature Analysis to Aid the Evaluation of CLASP (IMCS, August 
1994b) presented these findings. 

The research revealed no projects directly analogous to CLASP. It did find that many 
authors distinguished carefully between collaboration and cooperation. According to this 
scheme, collaboration involves mutual planning, a goal that neither institution could 
achieve alone, joint acquisition of funds, formal channels of communication, shared 
management and control, and equivalent contributions of time and resources. In 
contrast, cooperation involves short-term work with limited goals, one organization 
providing leadership and obtaining funds, occasional sharing of information, and 
benefits accruing primarily to one institution. 

There was remarkable consistency in the literature (which included the fields of library 
science, education, and, to a lesser extent, public policy and business) regarding success 
factors and barriers in collaborative relationships. Common success factors included: 

• A planning process that involves all partners equally and addresses 
accountability and evaluation; 

• Goals which are realistic, measurable, and mutually beneficial to all 
partners; 

• Demonstrated and consistent commitment of top leadership; 

• An institutional climate that supports change; 

• Positive attitude among individual staff and lack of jealousy; 

• Mutual respect and trust among partner organizations and individuals; 

• Active and equal participation by members of all institutions at all levels; 
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• Mid-level management that can work effectively with both top 
administrators and practitioners; 

• Adequate support by material and human resources; 

• Nurturing the relationship over time, though frequent feedback, public 
exposure, and acknowledgment of staff; and 

• Excellent communication and the provision of formal channels of 
communication. 

In addition, the study found that analyses of library-school cooperation typically cited 
lack of time, money, and st aff as the most significant barriers. Negative attitude, 
including resentment and jealousy between school and public librarians, also appeared 
as a significant impediment to cooperation in this context. 

The report concluded that while CLASP staff were successful in creating and making 
use of informal relationships with the schools, the relationship was one-sided. Schools 
did little more than participate passively in planning and goal setting. This failure to 
"own" the project affected top level management's involvement with long term concerns 
as well as individual staff members' contributions to programming. A fundamental 
barrier was the fact that material resources were allotted only to the library side of the 
relationship, and it was unclear what the schools themselves were expected to gain 
from the project. Finally, failure to establish evaluation measures and standards and 
unclear expectations for the stakeholders created confusion about accountability and 
project authority. These factors, as well as difficult institutional conditions, seemed to 
contribute to an overall indifference on the part of the schools and to the continuing 
difficulty CLASP had in overcoming barriers within the schools. 

Workshops 

The evaluation team designed and administered a number of workshops for CLASP 
and NYPL staff broadly related to key project themes and activities. 

Innovative Programming for Youth 

T=J — * 

Sari Feldman, Head of the Onondaga County Public Library's Central Library and 
expert on library programs and services for children and young adults gave a workshop 
on May 5, 1994. The goal of the workshop was to present ideas for innovative 
programming, grounded in child development theory, that could be incorporated into 
the 1994 CLASP summer program and future CLASP endeavors. Ms. Feldman 
communicated frequently with the CLASP Director about content and method while 
preparing the workshop. 
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Workshop content included: 



• A brief overview of the stages of child development, including the types of 
tasks preferred by children at each stage; 

• A comparison of two types of activities: interactive activities and 
independent learning activities; and 

• A range and variety of ideas for both interactive and independent learning 
activities for individuals and groups, with each idea described in the context 
of developmental stages. 

CLASP librarians had several opportunities to share their personal anecdotes about 
innovative CLASP activities and successful experiences with children. Ms. Feldman 
provided each participant with a bibliography of resources for innovative programming 
ideas, as well as copies of the New York State summer reading program manual, which 
contained hundreds of ideas for activities related to the summer's reading theme, 
"Reading Around the Clock." 

Linking Programs to the Curriculum Through Curriculum Mapping 

Dr. Michael Eisenberg, Professor, School of Information Studies, Syracuse University, 
and member of the IMCS evaluation team, presented a workshop on September 2, 1994. 
The purpose was to focus on how to link CLASP activities and services directly with the 
school's curriculum. 

Workshop content included: 

Review of the CLASP agenda including specific activities and services; 

Explanation of the curriculum mapping process, a technique for 
systematically gathering information on school curriculum; 

Practice in using the curriculum mapping technique; 

Discussion of curriculum attributes that are relevant to CLASP; 

Analysis of actual curriculum maps compiled from CLASP school districts 
to determine prime candidates for collaboration; and 

Discussion of tools for planning and documenting activities coordinated 
with classroom units. 

The workshop was highly interactive and combined conceptual discussion with skills 
development. Some attendees expressed a concern with the feasibility of applying the 
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strategies presented to their situations as CLASP libraries, since they must pit vide 
service to so many schools. 

Library Program Evaluation, Techniques and Issues 

A workshop for CLASP staff on September 27, 1994 discussed the importance and use 
of evaluation in assessing library programs. Presented by Dr. Charles McClure, 
Distinguished Professor, Syracuse University School of Information Studies and 
member of the IMCS evaluation team, it provided an overview of the evaluation 
process, its relationship to strategic planning, and an overview of specific evaluation 
techniques that can be implemented in the public library setting. Dr. McClure also 
discussed the development of institutional data collection activities and management of 
information for program evaluation. 

Specific objective:, of the workshop included enabling the participants to: 

• Describe the importance and role of evaluation of library services as part of 
the management process; 

• Describe step-by-step procedures for evaluation of various aspects of library 
services; 

• Describe the use and applications of various performance measures for 
library and information centers; and 

• Understand the relationships between maintaining organizational 
descriptive statistics — in some type of a management information system — 
and conducting program evaluations. 

McClure gave specific attention to developing evaluation strategies that could be used 
by CLASP during their current project activities. 

Collaboration 

The final workshop, on institutional collaboration, took place on October 7, 1994. 
Presenters were Dr. Ruth Small, Assistant Professor, Syracuse University School of 
Information Studies, and Dr. Jeffrey Katzer, Professor, Syracuse University School of 
Information Studies. Both are also members of the IMCS evaluation team. The 
workshop began with a brainstorming session in which participants described the 
characteristics of a "collaborative" relationship. The presenters then described the 
literature review process (see In-depth Study of School-Library Collaboration, above), 
including how collaboration was defined, the differences between collaboration and 
cooperation, and the dimensions of, critical success for, and barriers to collaboration. 
The presenters then described the results of IMCS' evaluation of collaboration within 
CLASP. The workshop concluded with a brainstorming session on ways in which 
CLASP could take the initiative to promote collaboration within the existing resources 
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(e.g., the need to develop better procedures for measuring what they're doing using the 
language and standards of outsiders). 

Overview 

Overall, these workshops provided CLASP/NYPL staff with an opportunity to consider 
a range of key issues and skills that could contribute to the ongoing success of CLASP 
activities during 1994-1995. A number of the participants commented to the instructors 
that the workshops provided them with information and insights that they could 
incorporate in future CLASP activities. 
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FINAL CONCLUSIONS AND RECOMMENDATIONS 

During the period from February 1993 through October 1994, CLASP made excellent 
progress towards achieving the first project goal, establish library use. For the second 
goal — organize cooperative activities with schools and parents — some notable progress 
was evident, but structural difficulties in creating top-level connections with the schools 
hampered efforts. Less was achieved in terms of formally achieving the third goal 
(develop models of cooperative service). 

In general, CLASP was highly successful in extending library programs and services to 
adults and children in Districts 6 (upper Manhattan), 8 (Bronx), and 31 (Staten Island) 
who might not otherwise have visited the library, attained a library card, or been 
exposed to public library offerings. CLASP was remarkable in being the first project to 
involve cooperation between a large urban public library system and public schools in a 
sustained and systematic manner. CLASP's ability to secure funding to carry it beyond 
its pilot phase attests to its success and its potential to empower libraries and schools in 
their mission to create a population of literate children and adults. 

As CLASP evolves out of its pilot phase and expands its services and its reach to other 
districts, it will need to attend to these points if it is to achieve project objectives 
successfully. These conclusions may be useful to other library systems, school systems, 
and grantors who may wish to implement similar programs in other cities. 

Conclusions 

• CLASP programs reached thousands of adults and children who might not 
otherwise have attended the library, registered for a library card, or been 
exposed to public library services. This was due to: CLASP's ability to offer 
many more programs than libraries had previously been able to offer; these 
programs being offered in schools and other locations where underserved 
populations could be reached; and an increased variety of programming 
that would appeal to targeted populations. 

• The impact of CLASP . peared to center on the number of activities offered, 
rather than the type. Similarly, librarians considered the ability of CLASP to 
provide great numbers of books and other resources not otherwise available 
to the branch libraries to be one of the most important benefits of the 
program. 

• CLASP/NYPL succeeded in raising schools' and the larger community's 
awareness of the library and in bringing new adult and child users into the 
library. Users reported a high level of satisfaction with CLASP programs, 
and there was evidence of increased library use through additional activities 
and resources provided by CLASP. 
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Teachers who were familiar with CLASP were enthusiastic about the 
program, cooperated in arranging and implementing activities, and tended 
to request repeat activities for their students. There was little evidence, 
however, that teachers were systematically involved in planning CLASP 
activities or that CLASP created links to school curriculum. 

All audiences judged CLASP activities to be very satisfying and interesting. 
This was true for all activities, all facets of satisfaction considered, and all 
audiences surveyed, and it was corroborated by teachers' and librarians' 
observations of the reactions of participants. 

CLASP activities did not appear to be any more satisfying for participants 
than non-CLASP activities. The greatest observable difference in 
satisfaction resulting from CLASP was on the part of NYPL librarians, who 
appreciated the increase in books and other resources funded by CLASP, as 
well as the presence of CLASP librarians to supplement their staff. 

Participants believed CLASP activities were beneficial in terms of providing 
information that would be useful at home, school, or work. Teachers, 
librarians, and parents felt the programs were educational, stimulated 
children to use the library, and encouraged reading for pleasure. To the 
extent that benefits could be assessed, CLASP activities did not appear to be 
any more beneficial than non-CLASP programs. 

CLASP succeeded in bringing many people of various ethnic backgrounds 
and different ages to library programs, especially when those programs 
were targeted specifically to those groups. Many disadvantaged people 
were introduced to the library through outreach to schools and other 
community sites. CLASP also had a positive impact on the equitable 
availability of resources across districts and communities within districts. 

Many of the programs, while they did attract underserved people to the 
library, did not create links to books, reading, or library services, raising the 
question of whether CLASP was taking full advantage of the opportunity to 
help people become regular library users. 

Both schools and libraries perceived an increase in school-library 
collaboration on the activity level as a result of CLASP, as well as improved 
rapport with the schools. However, collaboration usually emphasized joint 
scheduling rather than planning and goal setting, design, delivery, and 
evaluation of activities. Schools were often passive recipients of CLASP 
programs rather than active partners. 

There was no clear difference between CLASP and non-CLASP libraries in 
terms of the nature or success of their collaborative efforts. CLASP 
librarians made few attempts to link their services to school curriculum. 
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• There was disagreement about what constitutes "innovation." Many felt 
that an activity that is new or different is sufficiently innovative; others 
believed that "innovativeness" is useful only when applied to achieve a 
specific goal, such as increasing library use among a target group of 
underserved people or enriching school curriculum. 

• Many people, not only the public but also most school personnel and even 
some librarians, were largely unaware of CLASP and did not associate 
CLASP-sponsored activities with the program. 

Recommendations 

• CLASP should continue efforts to improve its visibility through public 
relations campaigns, written accounts in professional journals, and 
consistent identification of all CLASP-funded activities and materials. 
CLASP librarians should actively promote their efforts and successes, both 
to strengthen visibility and credibility, and to win the support of key 
members of the community. CLASP should secure the support of top 
school officials, top library administration, and even city officials. 

• CLASP should develop a better understanding of the information needs and 
interests of teachers and students in order to develop services and select 
resources that directly supplement established educational goals. 
Librarians may wish to employ a method such as curriculum mapping to 
systematically gather information about curriculum and then use such 
information in designing programs that are relevant to what is being taught 
in schools. 

• CLASP must ensure that collaboration goes beyond joint scheduling; they 
must work side-by-side with teachers in setting a program direction and 
planning activities that reflect joint planning, mutual benefit, and relevance 
to all audiences. Librarians and teachers should work together to bridge the 
gap between these two closely allied professions with an eye toward their 
mutual goal: improving literacy among youth. 

• CLASP programs and activities should support the educational goals of 
CLaSP by relating to the reading and information needs of youth, teens, 
and adults. If CLASP is to improve literacy and increase library use among 
its target audiences, programs should not only be interesting, motivating, 
and satisfying; they must also make some connection to books, literature, 
information, and the ongoing services and resources of the library. 
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MOVING FORWARD 

The CLASP effort has been an important one in learning how to link libraries, schools, 
and local communities together for new and better social/ educational services. 
Ultimately, solving urban social problems requires local strategies, with local agencies 
working together and coordinating their efforts. Such efforts, however, must have clear 
and realistic objectives, adequate resources, and dedicated staff and administrators. As 
shown in this project, libraries and schools can make significant contributions to 
strengthening local communities, promoting education, and offering a range of support 
to the community not otherwise available. 

The support of DeWitt Wallace-Reader's Digest Fund for CLASP activities resulted in a 
number of direct benefits and impacts described throughout this report. The effort also 
demonstrated the importance of such a project to the New York City government and 
resulted in ongoing resource support to the New York Public Library to continue 
CLASP. As such, CLASP can be seen as an ongoing experiment in how best to provide 
street-level involvement in the community by local libraries and schools. 

The formal evaluation of CLASP by Information Management Consultant Services, Inc., 
resulted in a number of important findings and lessons for future efforts. Funding 
agencies wishing to work with local governments, schools, and libraries in similar 
projects need to be aware of a range of issues related to collaboration among agencies 
such as those that participated in CLASP. Libraries and schools can also learn from the 
experiences of CLASP as they work to implement community-based educational 
programs. As suggested earlier in this report, understanding the political context in 
which the project will operate, providing incentives and rewards to participating 
agencies, and obtaining ongoing feedback and assessment of project activities are 
essential if collaboration is to be successful. Equally important is the involvement of 
dedicated and committed staff, such as the CLASP librarians, who indicated very 
clearly that they cared about the local community and wanted to improve it. 

The project also identified the complexity of successful collaborations among local 
institutions such as schools, libraries, community groups, and government. 
Successfully leveraging resources to affect multiple organizations at the local level is a 
critical area for ongoing developmental work. Lessons from the CLASP effort should be 
integrated into future collaborative projects. Funding sources such as the DeWitt 
Wallace-Reader's Digest Fund should continue to support efforts such as CLASP. 
Indeed, support for projects such as CLASP is essential if individuals in local 
communities are to be productive members of society. 

Efforts such as the CLASP initiative must continue. Libraries are a key institution to 
serve both as a community catalyst and service provider in improving educational and 
social services in the local community. As shown in this project, library programs can 
become a significant link between community members, the schools, local government, 
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and a range of educational opportunities which otherwise would not be available to the 
public. 

As initiatives such as CLASP continue to evolve, librarians, local officials, and fundors 
will continue to learn how best to meet the needs of the community and empower 
community members to be successful and productive members of society. Despite the 
willingness of many libraries to serve in this role, they require resources, training, and 
political support from local government to be successful. The project clearly 
demonstrated the importance of the library in providing a range of community-based 
programs and services. 
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APPENDIX: METHODOLOGY 

Dimensions of Study 

The three objectives guiding the evaluation included objectives relating to public 
libraries and public schools; to children and their families; and to public libraries and 
their communities (see pages 5-6). IMCS used the dimensions, or themes, defined 
below to define and assess the objectives. All dimensions did not apply to each 
evaluation activity or each period of study. Rather, the specific objectives of each 
evaluation determined the choice of dimensions. Dimensions studied at some point in 
the evaluation period include the following: 

• Extensiveness: Number, type, and locations of CLASP activities and 
attendance at those activities. (Studied in summer 1993, fall 1993, and 
summer 1994.) 

• Satisfaction: Participants' affective reactions to CLASP activities, including 
the degree to which they believed CLASP activities were meeting their 
needs. (Studied in summer 1993 and summer 1994.) 

• Benefit: The way(s) in which CLASP activities contributed to the quality of 
life for participants. (Studied in summer 1993 and summer 1994.) 

• Effectiveness: Extent to which the activity achieved the goals of CLASP. 
(Studied in summer 1993 and fall 1993. For fall 1993, this dimension 
included participants' satisfaction — their affective reactions; and benefits — 
the effects of CLASP on the lives of the participants. For the summer 1993 
and summer 1994 evaluations, satisfaction and benefits were considered 
separately.) 

• Management: Extent to which CLASP was well managed and well 
marketed. This theme included the decision making process, the 
organizational structure, the delineated authority and responsibilities 
within the organization, the cost-effectiveness of the program, and 
documentation supporting the structure. (Studied in summer 1993 and fall 
1993.) 

• Improvement: Ways to maintain, correct, or improve CLASP procedures or 
activities, from respondents' point of view. (Studied in summer 1993 and fall 
1993.) 

• Critical Success Factors: Extent to which the presence /absence of a factor 
determined the success/ failure of an activity, from respondents' point of view. 
(Studied in fall 1993.) 
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• Innovativeness: Extent to which CLASP instituted new approaches or new 
programs. (Studied in fall 1993 and summer 1994. For summer 1994, this 
dimension was considered under the scope of "improvement.") 

• Collaboration: Extent to which CLASP staff worked together with teachers, 
administrators, parents, and community organizations to further the goals 
of the project for the benefit of their mutual clientele. The collaboration 
dimension included the diversity of participants, decision-making authority 
within respective organizations, institutional linkages, and effectiveness of 
communication among members. (Studied in fall 1993 and summer 1994. 
For summer 1994, this dimension was considered under the scope of 
"improvement.") 

• Diversity: Extent to which CLASP programs drew audiences 
representative of the service districts, particularly uuuerserved populations 
including older children, at-risk students, people of various ethnic 
backgrounds, and people of different ages. (Introduced for the summer 
1994 evaluation, and considered under the scope of "improvement." ) 

Evaluation Design 

During the course of the overall evaluation, the IMCS team gathered data on a range of 
CLASP activities from a variety of sources, including school program administrators, 
CLASP and NYPL managers, students, parents, teachers, librarians, CLASP staff, public 
librarians, school librarians, and community members. Focus groups, questionnaires, 
individual and group interviews, and observations offered multiple data collection 
measures. 

IMCS utilized many data collection methods including broad-scale, focused-scale, and 
micro-scale approaches, defined below. To gather longitudinal data and allow for 
comparison, the team used control groups, repeated measures, and designation of 
capacity-demonstrating activities (i.e. "improved" activities selected for observation by 
the CLASP Director, summer 1994). 

1. Broad-scale Approach : Used to economically obtain data from a large 
number of people or organizations. This method employs a breadth of 
coverage at the expense of depth, meaning that broad-scale approaches 
often do not contribute much to a "rich" understanding of the phenomenon 
under investigation. 

Evaluators conducted telephone surveys to determine the knowledge of and 
perception of CLASP, the extent of collaboration between school librarians 
and CLASP staff, the attendance at CLASP programs, and the assessment of 
their effects. Researchers analyzed existing institutional records, activity 
logs, and circulation data for information related to attendance and use of 
materials. 
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2. Focused-scale Approach : Used to capture a more thorough and insightful 
appraisal of the phenomenon under consideration. Focused-scale methods, 
which are costly in time, provide information about how respondents react 
to and assess their interactions with the phenomenon under investigation. 

Researchers conducted group and individual interviews and also observed 
meetings and programs to gather in-depth data from participants in CLASP 
activities, advisory committees, and management. 

3. Micro-scale Approach : Provides the individual case studies of unique or 
important successes or failures; this approach contributes to the immediate 
improvement of the project as well as longer term transferability. 

Unique and /or important cases of success or failure required an in-depth 
approach. An examination of programs or activities which worked 
particularly well, or were less successful, contributed exemplars for 
consideration in the development of future CLASP programs. 

Sampling 

Summer 1993 . The summer program served approximately 40 sites, including public 
libraries, summer primary and intermediate schools, summer camps, and a variety of 
community organizations (see Table 4, below). IMCS evaluators distributed user 
surveys to CLASP staff with directions for administering them to CLASP participants. 
Researchers selected key individuals (e.g. librarians, teachers, community agency 
personnel, CLASP staff) and groups participating in CLASP activities for both in-person 
and telephone interviews. Evaluators observed 15 CLASP activities taking place 
approximately mid-way through the summer program. 

The CLASP Director helped identify six sites as comparable to CLASP libraries so that 
researchers could collect data for control comparisons. The sites included two in both 
the Bronx and Manhattan, and two in another borough. (It was necessary to select from 
another borough because all Staten Island libraries participated in CLASP.) 
Recommendations emerging from interviews lead to the selection of 16 program 
activities for micro-evaluation (i.e. in-depth examination of selected programs to 
determine reasons for success or areas needing improvement). 

Fall 1993 . The fall 1993 CLASP program held activities at community schools, at public 
libraries, and at a variety of community organizations (see Table 4, below). Selected 
individuals (students, teachers, parents, librarians, service providers, etc.) and groups 
participated in in-person interviews or telephone interviews after attending activities. 

Evaluators contacted three public librarians who had participated in telephone 
interviews during the summer 1993 evaluation for a longitudinal survey in fall 1993. 
Evaluators also conducted telephone interviews with nine school librarians /cluster 
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teachers on the basis of their familiarity with CLASP activities or mini-grants (i.e. 
awards given to teachers in public or non-public schools to conduct special programs 
consistent with the goals of CLASP). The evaluation also included seven micro-case 
evaluations selected from recommendations elicited from telephone interviews and on- 
site observations. 
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* Management focus group interview included all three districts 



Table 4. Evaluation Activities by Evaluation Period and District 

Telephone interviews with three school librarians and three public librarians who were 
not affiliated with the CLASP program provided comparative data. These librarians 
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were from districts in the Bronx selected because they generally matched the 
characteristics of each CLASP district. To obtain longitudinal data, evaluators 
interviewed a control group of public librarians who had participated in summer 1993 
telephone interviews. 

The evaluation included three focus group interviews conducted at CLASP advisory 
committee meetings (one per district), as well as school program administrator 
interviews from District 8 and District 31. 

Evaluators observed ten CLASP activities presented in October and November of 1993. 
These observations included parent workshops, CLASP librarian visits to classes and 
community organizations, an after-school program, and a mini-grant-sponsored school 
symposium. Altogether, the IMCS team observed four activities in Staten Island, four in 
northern Manhattan, and two in the Bronx. 

Summer 1994 . During one week of on-site evaluation, the IMCS team observed 13 
activities (see Table 4.) The CLASP Director designated six "capacity" activities (two 
from each district) as exemplars of innovativeness, collaboration, and /or diversity 
which would demonstrate CLASP's capacity to improve based on past 
recommendations. IMCS selected seven other "non-capacity" activities (i.e. "ordinary" 
activities similar to those offered in the past) based on district and type of activity, with 
an eye toward providing the richest and broadest representation possible during the on- 
site evaluation period. Altogether, the team observed two capacity activities and one 
non-capacity activity in District 6; two capacity activities and three non-capacity 
activities in District 8; and two capacity activities and three non-capacity activities in 
District 31. 

Instruments 

IMCS designed ten data collection instruments at the start of the evaluation. They also 
designed other instruments but held them in draft form until the experiences of the first 
formal evaluation (summer 1993) could be applied and the instruments refined. The 
initial instruments, as defined in Report #2: Instruments, Site Selection, Monthly Reports 
(IMCS, June 18, 1993), included: 

1. Mail Survey of Activities, Youth Version (English and Spanisn); 

2. Mail Survey of Activities, Adult Version (English and Spanish); 

3. Phone Survey, Participating (CLASP) Institutions; 

4. Phone Survey, Control Institutions; 

5. Phone Survey, Neighborhood (English and Spanish); 

6. Interview Guide, Staff Version; 

7. Interview Guide, Patron Version (English and Spanish); 

8. Interview Guide, Neighborhood (English and Spanish); 

9. Observation Guide, Activities; and 

10. Micro-Evaluation (Cases) — Data Fields Description. 
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As the evaluation evolved, IMCS revised existing instruments and designed new 
instruments in order to achieve the objectives of each evaluation period. New 
instruments included: 

11. Focus Group, Adult Version; 

12. Focus Group, Youth Version; 

13. User Survey, Adult Version (English and Spanish); 

14. User Survey, Youth Version (English and Spanish); 

15. CLASP/NYPL Group Interview; 

16. Advisory Committee Focus Group Discussion Questions; 

17. Interview Guide, Advisory Group Follow-up; 

18. Interview Guide, School Administrator Version; 

19. Interview Guide, Teacher/ Administrator Version; 

20. Telephone Survey, Participating Schools; 

21. Telephone Survey, Non-participating Schools; 

22. Observation Guide, Advisory Committee Meetings; 

23. Advisory Group Questionnaire; and 

24. Interview Guides, Outside Talent. 



All instruments used during summer 1993, fall 1993, and summer 1994 fell into three 
basic categories: user surveys, interview guides, and observation guides. 

• User Surveys . User surveys assessed participants' opinions about the 
satisfaction, benefit, and effectiveness of CLASP programs. There were two 
versions of the survey, one for youth and one for adults. Both were 
available in both English and Spanish. The surveys were designed to be 
self-administered, and contained either "yes /no" questions (for youth) or a 
simple Likert-type scale (for adults). Because the youth had a wide range of 
reading abilities, CLASP staff or members of the evaluation team read the 
questions aloud. 

• Interview Guides . IMCS designed several different instruments to collect 
data through interviews. The interviews targeted staff at CLASP and non- 
CLASP settings, program participants, and management from both CLASP 
and NYPL. Personal and telephone interview guides included a set of 
scripted questions representing each dimension under study. Focus group 
interviews were held with: (a) small groups of individuals who had just 
participated in a CLASP activity; and (b) small groups of NYPL or CLASP 
administrators or advisory committee members. Interviewers asked 
participants to discuss their reaction to the programs they had just attended. 
They asked management and advisory committee members about cost- 
effectiveness, management, collaboration, etc. Finally, evaluators used 
micro-evaluation case study guides to interview people in depth about 
CLASP activities that were determined to be either highly successful or in 
need of improvement. 
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• Observation Guides . Evaluators used observation guides to record 

information relating to the study dimensions, or themes, for each activity 
observed. During summer 1993 and fall 1993, sites of observations often 
coincided with those where interviews were conducted. 

IMCS pre-tested all instruments, with both English- and Spanish-speaking populations 
when appropriate. They carried out the pre-testing in communities with children 
participating in similar summer public library programs, with parents of participants, 
and with public librarians in upstate New York and New Jersey. Most of the pre-testing 
occurred during the spring of 1993, before the first formal evaluation. The pretests 
resulted in changes to vocabulary and wording of items in the interview guides. IMCS 
also revised the observation guide as an easy-to-use checklist. Members of the IMCS 
evaluation team involved in data collection attended training sessions that emphasized 
interviewing and observation procedures and techniques. 

The evaluation team administered all instruments except the user surveys, which were 
designed to be readily understood and easily self-administered by users. Questions on 
all instruments related to one or more of the evaluation dimensions (see Tables 5, 6, and 
7, below). Copies of all data collection instruments used during the evaluation are 
appended to the individual evaluation reports (IMCS, September 30, 1993; IMCS, 
December 1993; IMCS, August 1994a). 

Summer 1993 . Instruments used for the summer 1993 evaluation included two user 
surveys, seven different interview guides (including the micro-evaluation case study 
guide), and the activity observation guide. In addition to the instruments listed below, 
monthly activity logs were used to collect extensiveness data. 
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Dimensions of Study: Summer 1993 
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Table 5. Data Collection Instruments by Dimension of Study, Summer 1993 



Fall 1993 . Sixteen instruments were used during the fall evaluation. Of these, seven 
were new and nine were modifications of summer evaluation instruments. Time 
constraints did not allow all questions to be asked in some evaluation activities. Several 
of the new instruments gathered information from CLASP and NYPL management and 
advisory committee members regarding cost-effectiveness, critical success factors, 
collaboration, and other "higher level" concerns. 
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Dimensions of Study: Fall 1993 
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Advisory Comm. 
Meeting Obs. Guide * 


This instrument was used to record observations on the agenda, purpose, and leadership displayed at 
the meeting; also to record the extent to which discussion was one-sided or two-sided, and to track the 
group's contribution to CLASP management and decision making. 


Activities Observa- 
tion Guide 


• 


• 


• 




• 




• 


Advisory Group 
Questionnaire* 


Used to collect demographic information on advisory group participants, and to follow up on questions 
which were not discussed in the advisory committee focus group interviews due to insufficient time. 



* Indicates new instrument 

t Extensiveness data were derived frcm activity logs kept by each district 



Table 6. Data Collection Instruments by Dimensions of Study, Fall 1993 

Summer 1994 . Evaluators collected data by observing activities, interviewing staff 
individually, and interviewing participants in focus groups. The evaluation team used 
revised versions of existing data collection instruments. These included interview 
guides for patrons (parents), for youth (in focus groups), and for staff; and activity 
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observation guides. IMCS designed separate versions of both the staff interview guide 
and the activity observation guide for use in capacity and non-capacity activities. In 
general, the only questions retained from the summer 1993 evaluation instruments 
related to satisfaction and benefit. All instruments included questions related to 
innovation, diversity, and collaboration — the primary focus of this evaluation effort. 

Activity records (Activity /Encounter Forms) for the month of July supplied 
extensiveness data. CLASP staff submitted these directly to IMCS. 





Dimensions of Study: Summer 1994 
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* Extensiveness data were derived from Activity/Encounter Forms 



Table 7. Data Collection Instruments by Dimensions of Study, Summer 1994 
Limitations and Constraints 

The most significant constraint encountered during the evaluation process was the 
timing and initiation of the evaluation effort. To be most effective, the evaluation 
component should have been formulated at the conception of the project and 
implemented in the first year. 

Summer 1993 . The major issue affecting the design and implementation of this 
evaluation was the fact that CLASP's program plan was developed and initiated more 
than one year before the selection of the evaluation team and development of an 
evaluation plan. This affected the design and implementation in terms of: 
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• Randomization . Evaluating an ongoing program prohibits a full- 
randomized field trial and, therefore, affects the types of conclusions that 
state that the program was the cause of the observed results. IMCS' use of 
data from "control" (i.e. non-CLASP) institutions compensated for this 
shortcoming in part. 

• Comparable data . Not all CLASP institutions collected comparable 
information or kept consistent records through the course of the program. 
In addition, some data were incomplete or missing. In Report #2: 
Instruments, Site Selection, Monthly Reports (June 18, 1993), IMCS 
recommended a more organized, consistent and comprehensive reporting 
format for the CLASP activity logs. The CLASP administration then agreed 
to adopt a similar format for programs beginning in the fall of 1993. 

• Site-based sampling . Because so many of the summer CLASP activities 
were held at non-library sites (e.g. schools, community agencies) and 
because may CLASP schools and libraries within each district did not 
conduct CLASP activities during the summer, sampling for the summer 
evaluation could not be site-based. Instead, sampling was based on 
representation of a range of activities sponsored in each district during the 
summer program. 

In addition, last minute planning and decision making for summer 1993 CLASP 
programs and activities resulted in considerable uncertainty for the evaluation team as 
to what programs would be offered, where they would be offered, or when they would 
be offered. This affected sampling and logistical decisions. 

Fall 1993 . Several constraints affected the design and implementation of data collection 
activities for fall 1993: 



The CLASP Director resigned late in the summer of 1993, and the position 
was vacant at the time evaluation activities needed to be scheduled. While 
other staff in the organization proved helpful, frequent rescheduling of 
activities and meetings made planning difficult. 

The delay in Hie opening of New York Public Schools in September 1993 
meant that the scheduling of fall data collection could not be done until late 
September. 

Scheduling school-based interviews proved difficult because desired 
respondents could not be contacted directly. 

Meetings and activities were frequently rescheduled, and some activities 
selected for observation were canceled with little or no notice. 
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• IMCS was unable to observe any teacher workshops due to schedule 
changes and cancellations. 

Summer 1994 . The summer 1994 data collection process was less comprehensive than 
the summer 1993 effort, primarily because the evaluation focused on the particular issue 
of improvement, rather than a more complete assessment. On-site observation was 
more limited in scope, and there were few difficulties in obtaining and analyzing data 
during the week on-site. One concern was that the extensiveness data might be 
incomplete because outsourced programs (i.e. programs given by non-library people 
hired to present them), which were not always attended by CLASP staff, appeared to 
have been inconsistently reported through Activity /Encounter Forms. 

In summary, IMCS conducted a study that was both broad and rich by using a wide 
range of data collection techniques, and by soliciting the views of youth and adult 
participants, librarians, teachers, school administrators, CLASP and NYPL 
administrators, community leaders, advisory committee members, and others in the 
evaluation process. Because the original methodology allowed fov expansion and 
revision, IMCS was able to alter the study method as needed to reflect the primary 
concerns of DeWitt Wallace-Reader's Digest Fund during each evaluation period. 
Being brought "on-board" over one year into the program was a handicap in designing 
an evaluation methodology; IMCS compensated for this as well as possible and 
collected data which reflected the impact of CLASP and the program's success in 
reaching its three original goals. 



